Evaluating extracted phrases and extending thesauri

We describe an interface that uses the phrases occurring in a document collection as a basis for browsing the collection and accessing its contents. Phrases are automatically extracted from the document text to represent the subject matter of the collection. Clearly, the interface’s utility depends on how good these phrases are. We evaluate the system by comparing the phrases extracted from a large Web site to those in a thesaurus used by the organization responsible for the site. This analysis serves two purposes: it aids the user by verifying that the phrases extracted are relevant to, and provide good coverage of, the subject areas of the Web site and thesaurus; and it aids the thesaurus compiler by identifying phrases in widespread use that do not appear in the thesaurus.